-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
async_hooks: add AsyncLocal API mk2 #31016
Conversation
4a777bf
to
8097644
Compare
An interesting update. I've implemented benchmark based on |
@mcollina @benjamingr |
I really like this implementation. |
b191b81
to
498c40e
Compare
As an optimization suggested by @Qard (see nodejs/diagnostics#345 (comment)), async hook per Public API and behavior of |
Another interesting update. I've compared performance with |
lib/async_hooks.js
Outdated
if (!this[kResToValSymbol]) { | ||
throw new ERR_ASYNC_LOCAL_CANNOT_SET_VALUE(); | ||
} | ||
this[kResToValSymbol].set(executionAsyncResource(), value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would set a barrier between the current execution layer and its parent, discarding the upper value and only propagating the new one downward through the tree. That's not necessarily a bad thing, but a user could very easily make the mistake of calling set
again somewhere further into the request thinking it would allow a different path through the request to access the value, which is not the case. In my experience, it's better for the barrier to be applied in the constructor that way the object can't unexpectedly fork part-way through a request. Rather than constructing one AsyncLocal
instance at the top-level, you'd construct a new one at the start of each request.
I know there's also the use-case of enabling isolated components of a system to share data without explicit communication, and I think the best way to achieve that is just using a factory that can be called to produce or retrieve AsyncLocal
instances. In a way, you kind of already have this with the set
and get
, but I feel the naming is misleading and should be made clearer that set
is forking the following async tree to use the new context object.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than constructing one AsyncLocal instance at the top-level, you'd construct a new one at the start of each request.
With this API such usage should be considered a bad practice, because of performance reasons and tricky API manipulations. By the latter I mean that it's hard to share individual AsyncLocal
instance with each request. It's much simpler to use a single instance to handle all requests. See this example: https://github.com/nodejs/node/pull/31016/files#diff-9a4649f1c3f167b0da2c95fc38953a1fR485
As for merging values or doing any other actions on value modification, that could (and, in my opinion, should) be done in user-land, i.e. in application code or libraries, because exact logic depends on particular use case.
In a way, you kind of already have this with the set and get, but I feel the naming is misleading and should be made clearer that set is forking the following async tree to use the new context object.
I think that .get()
/.set(value)
method names are quite intuitive for anyone who is familiar with the concept of asynchronous call chain. To help users, PR adds a couple of examples showing how values are propagated into the documentation.
For me, alternative method names, like .read()
/.fork(value)
or something like that are much less obvious.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel the opposite. The .get()
and .set(value)
names are not intuitive at all to someone trying to use the API to solve the need of having a unique context object per-request. It's capable of achieving that by using .set(value)
at the start of the request to store an object and then only using .get()
to retrieve and modify that object for the rest of the request, but that's very non-obvious from the naming or the docs. The only reason it makes sense to us is because we are already intimately familiar with the concept of async call trees and how this feature is implemented. An average user is not, and I'm almost 100% certain users will try to use this like ThreadLocal in Java and then be totally confused when different parts of their app get different data because they keep calling .set(value)
to "update" the stored value; it looks a lot like setState(...)
in React, and I suspect many people will treat it as such. I'm sure many users will also try to construct a new AsyncLocal within each request as that's a common practice with ThreadLocal in Java, again doing it the wrong way here.
In my opinion, a proper async context API should look something more like this:
// This corresponds roughly to the existing `AsyncLocal`
const contexts = new ContextManager()
function loadUser(auth) {
const { onError, onUser } = contexts.current
User.login(auth).then(onUser, onError)
}
function httpResponder(res) {
contexts.current.onError = err => {
res.status(500).end(err.message)
}
contexts.current.onUser = user => {
res.status(200).end(user.token)
}
}
app.post('/login', (req, res) => {
// This corresponds to `.set(value)` and is closest
// to what most people actually think of as a local.
contexts.create()
httpResponder(res)
loadUser(req.headers.authorization)
})
It's functionally identical to what you have built, but more cleanly communicates the intent of the objects and methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The snippet looks like something that can be built on top of AsyncLocal
API, but personally I find it less straightforward. AsyncLocal
API, in its current state, allows building APIs on top of it, say, the one from the snippet or something similar to cls-hooked
/continuation-local-storage
API. So, I expect it to be mainly used by library authors, not all node users.
As for user confusion, I hope that it can be mitigated by good enough documentation.
lib/async_hooks.js
Outdated
remove() { | ||
if (this[kResToValSymbol]) { | ||
delete this[kResToValSymbol]; | ||
locals.delete(this); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really like the manual cleanup required here. If someone decides to make a unique AsyncLocal
object per-request it may be ambiguous where to close the object and they might even not close it at all, leading to a memory leak. It'd be better to use a WeakSet so you don't have to worry about explicit cleanup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WeakSet
won't do here, as it doesn't allow iteration over its items. A Set
with WeakRef
s + FinalizationGroup
API could potentially help to achieve automatic AsyncLocal
clean up, but FG API is experimental and has to be enabled via a flag.
On the other hand, other CLS API implementations do not provide a way to free underlying resources and disable hooks. That's because most applications use a single instance of CLS without getting rid of it. So, AsyncLocal.remove()
method is an advanced API, useful in specific scenarios.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's ways to eliminate the global Set
entirely by making the WeakMap
global and propagating on that directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A global WeakMap
used to store values won't allow to isolate different AsyncLocal
s, which kills the whole idea.
If you mean a WeakMap
that stores <AsyncLocal, WeakMap>
pairs, then it's not possible to iterate over WeakMap
entries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean something like WeakMap<AsyncResource, Set<AsyncLocal>>
. The hooks can loop over the set of AsyncLocal instances at the current level and propagate them downwards. It means an array at each level, rather than just the top-level, but it also cleans up automatically.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's an interesting idea. Another level of indirection with a WeakMap<AsyncResource, Set<AsyncLocal>>
will help to clean up both values and AsyncLocal
s list in the hook. As downsides I can see the following:
- The hook will have to be kept always enabled, once the first
AsyncLocal
instance was created. Currently.remove()
disables the hook if there are no activeAsyncLocal
s. - This change will probably have a certain performance impact. But that has to be measured.
In general, it sounds like a nice idea for an experiment. I'm going to do that experiment, if this PR gets some chances to be reviewed and merged (I understand that chances are close to zero, especially considering #30959 (comment)).
498c40e
to
77030da
Compare
c87b7ea
to
c5d938d
Compare
9a2b28c
to
4039d21
Compare
11f22e9
to
8c488a9
Compare
As #31746 was created recently, it makes no sense to have this PR closed. So, I'm reopening it. |
67a1890
to
7619e7b
Compare
@puzpuzpuz No problem. I don't care much about my authorship and this PR had already a link to the previous PR. |
7619e7b
to
c903037
Compare
Introduces new AsyncLocal API to provide capabilities for building continuation local storage on top of it. The implementation is based on async hooks. Public API is inspired by ThreadLocal class in Java. Co-authored-by: Gerhard Stoebich <[email protected]>
c903037
to
2406a9a
Compare
@Flarna looks like I have addressed all of your points. Could you do another review round? |
Closing as #26540 has landed |
Depends on async_hooks: add executionAsyncResource #30959. See the last commit to understand what this PR changesbenchmark/async_hooks/async-resource-vs-destroy.js
)Introduces new AsyncLocal API to provide capabilities for building continuation local storage on top of it.
The implementation is based on
async hooks
(hook callbacks +executionAsyncResource()
), but async hooks are not exposed in public API in any way. Public API is inspired by ThreadLocal class in Java.I've marked @Flarna as a co-author, as the idea of this PR came out of #27172.
FYI @Quard, @Flarna, @vdeturckheim
Design overview
Main properties of the new API:
.get()
/.set()
+ optional.destroy()
methods)AsyncLocal
, say,continuation-local-storage
-like API orAsyncContext
(async-hooks: introduce async-storage API #26540). See sample snippets from the following comment: Executive summary: Introducing a CLS-like API to Node.js core TSC#807 (comment)destroy
hook. Thus, misbehavingAsyncResource
won't be able to lead to mem leaks.destroy()
method disables the hook callback and frees all values from memory. Thus, it's possible to remove all remains of theAsyncLocal
Benchmarks against alternatives
I have modified
async-resource-vs-destroy.js
benchmark and compared proposed implementation with #26540. Here is the result that I got on my machine:As expected,
AsyncLocal
is significantly faster thanAsyncContext
, but slower than the sample implementation that stores values as resource object properties (async-resource
type). As for the comparison with latter, I think that.destroy()
method that frees all values from the memory is more valuable than certain performance gain.Benchmarks against existing libraries
A performance comparison with
cls-hooked
v4.2.2 was also made. The benchmark is available here: https://gist.github.com/puzpuzpuz/f0b23458a821d7edab3738550e58f0e2 (again, it's a fragment ofasync-resource-vs-destroy.js
).Here is the result:
With these results
cls-hooked
shows worst results among all candidates. But if bothns.bindEmitter
calls are commented (which contradicts with library guidelines for middlewares), the result improves significantly:With this result,
cls-hooked
is on par withAsyncLocal
.Possible enhancements
WeakMap
That should bring
AsyncLocal
's performance toasync-resource
's level (see benchmark results above). The downside is that it won't be possible to free values in all reachable resources in.destroy()
method anymore, so this method will provide weaker guarantees after the change.With current implementation, if a strong reference to
AsyncLocal
is destroyed in application code, a strong reference inasync_hooks
will still remain and the value propagation will continue to be executed. Thus, the instance will be effectively leaked.Note: the same consideration also applies to
async_hooks.createHook
. Once the user-land reference to anAsyncHook
is lost, it's not possible to disable it.To make the API less error-prone
AsyncLocal
's local constructor could be kept private andcreateAsyncLocal(string)
/destroyAsyncLocal(string)
functions could be introduced toasync_hooks
. EachAsyncLocal
would be identified by a string (the exact type of id is not that important) and multiple calls ofcreateAsyncLocal
with the same identifier would return the sameAsyncLocal
instance.The main problem with this enhancement is user-land library isolation, i.e. library A shouldn't be able to retrieve library B's instance.
Alternatively, option 1 from comment #26540 (comment) can be considered. See this PR for draft implementation and benchmark results: https://github.com/vdeturckheim/node/pull/2
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes